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Maurice M. Tatsuoka and Robert E. Comley* 



Abstract 



Three statistical _nalyses were performed on test 
score data for students from Inglewood, California: (1) 
a matched pairs design (2) a covariance analysis, and 
(3) an item analysis in terms of difficulty indices. Forty 
matched-pairs of UICSM and non-UICSM students were 
formed on three variables and the two groups were compared 
on the Cooperative Elementary Algebra and Coop Math, 
Algebra 1, tests. No significant differences were found in 
test score means. The covariance analyses resulted in 
statistical adjustments on six and five variables for junior 
and senior high school groups of students respectively. 

The same two criterion variables were used for both the 
junior high and senior high groups and significant differences 
were found in both groups when adjustment was not made for 
teacher ratings by school principals. The results showed 
significantly higher achievement by the UICSM students on 
both the Coop Elementary Algebra and Coop Math exam- 
inations. The item analysis of the two criterion tests 
showed significantly better achievement by UICSM students 
on ten items from the two tests; while non-UICSM students 
excelled in four items at a significant level. 



The studies described in this report were initiated by the research section 
of UICSM at the request of Dr. Quentin R. Bryan, Coordinator of Special 
Projects for the Inglewood Unified School District. These studies consisted 
of tests administered by Inglewood teachers but selected and scheduled in 
close collaboration with the senior author, who visited the school system and 
maintained close contact wit!* Dr. Bryan and his staff. This request for help 
offered an opportunity for the project to obtain a “reading” on the level of 
achievement in a school system that had newly adopted the UICSM curriculum. 

The Inglewood Unified School District includes two junior high schools 
(7-8) and two senior high schools (9-12). Five classes of superior 8th grade 
students, one clasr of superior 9th grade students, and one class of “randomly* 
selected 9th grade students used UICSM materials as their course of study. 
Median intelligence scores for students in this district have consistently been 
above the national norm. Placements in the junior high school experimental 



*The authors wish especially to acknowledge the assistance of Judith Boyle 
who performed many of the calculations. 
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sections were made on the basis of the California Algebra Aptitude Test (CAA), 
administered in the spring of 1962, and the high school students were selected 
on the basis of past performance in mathematics. A total of 209 pupils were 
enrolled in these seven UICSM sections. Five of these sections (those at the 
8th grade level) had studied only the first ten pages in Unit 4, at the time the 
two Coop algebra tests were administered. The other two experimental sections 
(9th grade) had completed Unit 4. This means that approximately 150 out of 
the 209 UICSM students in these sections completed only about three -fourths of 
the First Course at the time of testing. 



Three studies have been completed since the inception of this collaboration 
between the Inglewood schools and UICSM. These include a comparison by a 
matched-pairs design, a total group comparison in which covariance analysis 
provided statistical control, and an item analysis of the tests which obtained 
difficulty indices for the UICSM experimental (E) group and the comparison (C) 
group on each item. 



V 



A Comparison of Matched Groups 



Forty first-year UICSM students (9th grade) were matched with forty non- 
UICSM 9th-grade students of first year algebra on three variables: (1) CTMM-L, 
(2) CTMM-NL,! and (3) STEP-Math. 2 The degree of matching may be noted in 
Table 1. At the end of the year, scores on the Cooperative Elementary Algebra 
Test, Form T(1950), and the Cooperative Mathematics Test, Algebra I, Form 
A(1962) were obtained for thirty- two of the forty pairs. The use of standardized 
tests of this character as criterion variables for comparison of achievements 
made by the two groups were justified by the fact that the UICSM course has the 
same general goals as traditional algebra. 



1CTMM: California Test of Mental Maturity (1957); (L, language; NL, 
non-language) published by the California Test Bureau. 

2 STEP-Math: Sequential Tests of Educational Progress — Mathematics 
(1957) published by the Cooperative Test Division, Educational Testing 
Service. 
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Table 1. Mean differences on three control variables and two criterion 

variables for matched pairs of UICSM and comparison students. 

Control Variables Criterion variables 









STEP 


Coop 


Coop 




CTMM-L 


CTMM-NL 


Math 


Algebra 


Math 


Mean differ- 












ence (E - C) t 


0 


.45 


. 50 


2.06 


-1.03 


S. E. of mean 












difference (E - C) 


O 

00 

• 


.46 


.32 


1.83 


1.04 


t 


0 


.69 


1. 56 


1. 13 


-0.99 


No. of pairs 


40 


40 


40 


32* 


32 


^Criterion scores 


were not available for all 40 


pairs 


of students. 





t Hence, positive differences are in favor of UICSM. 



As shown in Table 1, the results were rather equivocal and in a strange 
direction. The UICSM group did somewhat better than its counterpart on the 
Coop Algebra test, but the difference was not statistically significant as determined 
by a one-tailed t-test at the 5% level. The comparison group on the other hand 
had a non- significantly higher mean on the Coop Math, Algebra I, test. We are 
not entitled to draw any conclusion? However, we should note that the opposite 
finding might well have been expected, since the Coop Algebra test is heavy 
on applications which are less strongly emphasized in UICSM first course 

than in traditional algebra courses. On the other hand the new Coop Math, 

Algebra I, test contains a number of “modern mathematics’’ items, e. g. , 29, 

33, 35, 36, so one might have expected UICSM students to perform better on 
it. 






A comparison of all UICSM students v .c. all non-UICSM algebra students 

A more extensive analysis of the data on Ingle wood 8th and 9th grade students 
was carried out, using covariance techniques to apply statistical controls. The 
junior high school group (8th grade) contained 128 UICSM students X E) and 38 
Algebra I students (C). The text used by these non-UICSM students was 
Algebra One , Hayden ~nd Finon (1961). Covariance adjustments were made for 
the control variables SCAT-V, SCAT-Q, CAA, STEP-Math, sex, and teacher 
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rating as made by the principals of the schools. 3 An unrestricted linear hypothesis 
modti (using separate group regression weights) was used instead of the standard 
covariance analysis. 4 Data relevant to the distributions of scores on those tests 
are given in Table 2 . 



Table 2. Means and standard deviations of test scores for 
the UICSM- and non-UICSM groups of 8th grade 
students. 





UICSM 

Mean S. D. 


Non-UICSM 
Mean S. D. 


SCAT-V 


282. 30 


7. 78 


282. 55 


8.81 


SCAT-Q 


296. 22 


8. 19 


297. 39 


6.94 


CAA 


64.73 


11. 35 


64. 87 


8.67 


STEP-Math 


279. 22 


7.96 


280. 68 


6. 19 




n e 


- 128 


N C = 


38 



The criterion variables were the Coop Algebra Test (1950) and the Coop 
Math, Algebra I, Test (1962). The UICSM group mean scores exceeded the 
corresponding mean scores of the non-UICSM group, and the differences were 
statistically significant after covariance adjustments were made, if teacher 
rating was excluded, as is shown in Table 4. 

The senior high school group contained 38 UICSM students (E) and 171 
non-UICSM students (C). The control variables, for which covariance 
adjustments were made, include STEP-Math, CTMM-L, CTMM-NL, sex, 
and a teacher rating made by school principals. Means and standard deviations 
of scores on the control variables are given in Table 3. 



3 SCAT: Cooperative School and College Ability Tests (1955) 

(Q, quantitative; V, verbal) published by the educational 
Testing Service. 

CAA: California Algebra Aptitude Test (1958) published by the American 

Guidance Service, Inc. 

4 We are indebted to Dr. Frank Watson, now at the Cancer Research Hospital, 
Columbia, Missouri, for making available to us his then unpublished computer 
program for carrying out the test. 

o 

ERIC 



5 



Table 3. Means and standard deviations of scores of UICSM-and 
non-UICSM 9th grade students on the control tests. 

UICSM Non-UICSM 





Mean 


S. D. 


Mean 


S. D 


STEP-Math 


284. 92 


9.46 


280. 15 


9.06 


CTMM-L 


46.03 


7.42 


40. 96 


8. 32 


CTMM-NL 


33. 37 


5. 13 


30. 80 


5. 27 




n e 


= 38 


N c = 


171 



The criterion variables were Coop Algebra Test (1950) and the Coop Math, 
Algebra I, Test (1962). Again the means of the scores for the UICSM students 
exceeded the corresponding means of the control group; but after covariance 
adjustments for all of the control variables, the differences were not 
statistically significant. However, when covariance adjustment for teacher 
rating was omitted, the adjusted difference in mean scores on the Coop algebra 
test was 10.2 and was statistically significant at the 5% level. The results are 
recorded in the summary table below. It is interesting that the significant 
differences occurred when covariance adjustments were not made on teacher 
rating. This suggests that the superior performance of UICSM students may be 
due to their having superior teachers. Thus, either due to the self- selection 
factor (superior teachers wanting to teach UICSM) or to their having profited from 
the UICSM Summer Institute training (or both), it seems that UICSMt teachers do 
get a higher rating by principals, and that their superiority as teachers has a 
positive effect on student performance. 



Table 4 . Summary of Inglewood Results 
1 Junior High 



E = UICSM 



(N = 128) 



C = Algebra 1 



(N = 38) 



Criterion Group Means 
Variable E C 

Coop. Algebra 22.4 19.9 

Coop. Math 25. 4 23. 5 

^Control variables: 




Significance at 5% level when 
covariance adjustments were made on: 
All control vars.* All but teacher rating 

Not signif. Signif. 

Not signif. Signif. 

SCAT-V, SCAT-Q 

CAA, STEP- Math 
Sex, Teacher Rating 



UICSM 



b 



II 



Senior High 



E 

C 



T raditional 



(N = 38) 
(N = 171) 



Significance at 5% level when 

Criterion Group Means covariance adjustments were made on: 

Variable E C All control vars,* All but teacher rating 

Coop. Algebra 34. 5 24. 5 Not signif. Signif. 

C oop. Math 30. 0 25. 8 Not signif. Not signif. 

^Control variables: STEP Math 

CTMM-L, CTMM-NL 
Sex, Teacher Rating 



In order to permit some comparison of the students in Inglewood schools 
with students in the older pilot and participate * schools, a summary of data on 
the Coop Algebra examination for students who began study of UICSM Course I 
in 1958 and 1959 is included (Table 5). A detailed discussion related to these 
data has been given by Tatsuoka and Easley (1963). 



Table 5 . Coop Algebra means and standard deviations of raw scores for some 
groups of students in UICSM Course I and comparison groups in 1958 
and 1959. 



Dates of 


UICSM 






Non- UICSM 




Testing 


Means 


S. D. * 


N 


Means S. D. * 


N 


May 1959 


28. 58 


8 


118 


22. 35 12 


515 


Sept. -Dec. 1959 


26. 72 


9 


270 


20. 75 11 


161 


May I960 


22. 70 


12 


574 






Oct. -Nov. 1960 


27. 91 


9 


382 






Jan. 1961 


26. 18 


10 


135 






May 1960-Mar. 1961 


28. 89 


11 


226 







* Approximations of standard deviations of raw scores from standard deviations 
of scaled scores. 



It may be noted that the Inglewood Senior High UICSM students had a higher 
mean score on the Coop Algebra examination than the students in these compari- 
son groups. 
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Comparison of item difficulty indices for UICSM 
and non-UICSM students in Inglewood, California 

A third comparison of UICSM and non-UICSM students at Inglewood was 
made with respect to relative success on individual items on the Coop Algegra 
and Coop Math tests. The percentage of students correctly answering each item 
of the two criterion tests was computed. Such percentages are customarily 
called difficulty indices . The smaller the index, the greater is the difficulty 
of the item, i. e. , the smaller the number of students answering the item 
correctly. The calculations were based on answers by 209 UICSM students and 
249 non-UICSM students on the Coop Algebra test, and 187 UICSM and 249 
non-UICSM students on the Coop Math test. 

Some differences in success for the various test items were noted. Those 
items for which differences in difficulty indices for the experimental and 
control groups were found to exceed or equal 0. 20 (a significant difference 
at the 5% level) are listed below. 



A. Diff. Ind. 



— Diff. Ind. 



> 



correctly answering an item exceeded, by 20 
non-UICSM students who got the item right. ) 



0. 20 (The percentage of UICSM students 
or more, the percentage of 



1. Coop Algebra Test 

UICSM students did better on items 31, 3 5, and 37 on Part I and 

items 10, 11, and )6 on Part II of the Cooperative Algebra test, Form T 

(1950), published by the Educational Testing Service. The first group of 
these items deals with combining and simplifying algebraic fractions and 
the second group deals with the general concept of defining a new quantity 
as the quotient of two other quantities, e.g. . speed as the ratio of distance 
and time, trigonometric functions defined as ratios of particular sides of 
a triangle, and cost per article when given the cost of many articles. 

2. Coop Math Test 

UICSM students did significantly better on items 4, 27, 29, and 38 of 
The Cooperative Mathematics Test, Algebra I, Form A (1962) which is also 
published by ETS. The four items listed above treat knowledge of algebraic 
fractions, non-divisibility by zero, inequalities, and recognizing that a given 
linear equation has no solution. These are all topics on which UICSM 
materials place greater stress than conventional texts do. 

B. Diff. Ind. _ — Diff. Ind. _ ^ 0. 20 



1 . Coop Algebra Test 

The comparison group did better than UICSM students on items 9 
and 16 of Part I of the test. These items deal with the laws of exponents 
and graphical solution's of linear equations. (As of the time of testing, the 
UICSM students had not studied the laws of exponents or two-dimensional 
graphs of solution sets.) 
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2. Crop Math Test 

Items 7 and 9 of this test were answered better by non-UICSM student 
These items test understanding of the definition of coefficient and the laws 
of exponents. Again these topics had not been studied by UICSM students 
at the time of testing. 



'I' "I' 



A principal- axis factor analysis is presently being carried on in an attempt 
to classify items into several groups which seem to depend on different skills 
and knowledges. However, no conclusions have been reached at this point. 

Item analysis is also being carried on in connection with a UICSM elementary 
algebra inventory which covers many objectives not covered in the two Coop 
tests. 
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